Approximate Dynamic Programming: a $\mathcal{Q}$-Function Approach

In this paper we study both the value function and Q-function formulation of the Linear Programming (LP) approach to ADP. The approach selects from a restricted function space to fit an approximate solution to the true optimal Value function and Q-function. Working in the discrete-time, continuous-space setting, we extend and prove guarantees for the fitting error and online performance of the policy, providing tighter bounds. We provide also a condition that allows the Qfunction approach to be more efficiently formulated in many practical cases.